Benchmarking Optimization Software with Performance Profiles
نویسندگان
چکیده
We propose performance profiles — distribution functions for a performance metric — as a tool for benchmarking and comparing optimization software. We show that performance profiles combine the best features of other tools for performance evaluation.
منابع مشابه
A server for automated performance analysis of benchmarking data
As part of Performance World, we describe an automation server (PAVER: http://www.gams world.org/performance/paver) to help facilitate reproducible performance analysis of benchmarking data for optimization software. Although PAVER does not solve optimization problems, it automates the task of performance data analysis and visualization, taking into account various performance metrics. These in...
متن کاملOptimality Measures for Performance Profiles
We examine the influence of optimality measures on the benchmarking process, and show that scaling requirements lead to a convergence test for nonlinearly constrained solvers that uses a mixture of absolute and relative error measures. We show that this convergence test is well behaved at any point where the constraints satisfy the Mangasarian-Fromovitz constraint qualification and also avoids ...
متن کاملPAVER 2.0: an open source environment for automated performance analysis of benchmarking data
In this paper we describe PAVER 2.0, an environment (i.e. a process and a suite of tools supporting that process) for the automated performance analysis of benchmarking data. This new environment improves on its predecessor by addressing some of the shortcomings of the original PAVER [6] and extending its capabilities. The changes serve to further the original goals of PAVER (automation of the ...
متن کاملUsing Performance Profiles for the Analysis and Design of Benchmark Experiments
It is common to manipulate a large amount of data generated in the benchmarking process when comparing metaheuristics. Performance profiles are analytical tools for the visualization and interpretation of these results. Here we comment on their explanatory power, discuss novel variants, introduce a multicriterion view of the performance comparison, and also define performance profiles for each ...
متن کاملBenchmarking Derivative-Free Optimization Algorithms
We propose data profiles as a tool for analyzing the performance of derivativefree optimization solvers when there are constraints on the computational budget. We use performance and data profiles, together with a convergence test that measures the decrease in function value, to analyze the performance of three solvers on sets of smooth, noisy, and piecewise-smooth problems. Our results provide...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Math. Program.
دوره 91 شماره
صفحات -
تاریخ انتشار 2002